As is known in the art, it is sometimes desirable that the data storage capacity of the data storage system be expandable. More particularly, a customer may initially require a particular data storage capacity. As the customer's business expands, it would be desirable to correspondingly expand the data storage capacity of the purchased storage system.
Small Computer Systems Interface (“SCSI”) is a set of American National Standards Institute (“ANSI”) standard electronic interface specification that allow, for example, computers to communicate with peripheral hardware.
SCSI interface transports and commands are used to interconnect networks of storage devices with processing devices. For example, serial SCSI transport media and protocols such as Serial Attached SCSI (“SAS”) and Serial Advanced Technology Attachment (“SATA”) may be used in such networks. These applications are often referred to as storage networks. Those skilled in the art are familiar with SAS and SATA standards as well as other SCSI related specifications and standards. Information about such interfaces and commands is generally obtainable at the website http://www.t10.org. As used herein, reference to SAS devices and protocols may be understood to include SATA devices and protocols.
Such SCSI storage networks are often used in large storage systems having a plurality of disk drives to store data for organizations and/or businesses. The network architecture allows storage devices to be physically dispersed in an enterprise while continuing to directly support SCSI commands. This architecture allows for distribution of the storage components in an enterprise without the need for added overhead in converting storage requests from SCSI commands into other network commands and then back into lower level SCSI storage related commands.
A SAS network typically comprises one or more SAS initiators coupled to one or more SAS targets often via one or more SAS expanders. In general, as is common in all SCSI communications, SAS initiators initiate communications with SAS targets. The expanders expand the number of ports of a SAS network domain used to interconnect SAS initiators and SAS targets (collectively referred to as SAS devices or SAS device controllers).
In general, a SAS initiator directs information to a SAS target device through ports of one or more SAS expanders in the SAS domain. A “port” in SAS terminology is a logical concept. A port may comprise one or more physical links in a SAS domain. Such physical links are often referred to as phys in the terminology of SAS domains. A port may use a single phy or, if the port is configured as a wide port, may use multiple phys logically grouped to provide higher bandwidth. Each phy can support one SAS lane or channel.
In the SAS standard, a logical layer of protocols includes the phy. Each phy is configured for passing data between the SAS device and another device coupled thereto. As used herein, “physical layer”, “phy”, or “phy layer” all refer to a protocol layer that uses a physical transmission medium used for electronic communication including, but not limited to, the phy layer as specified in the SAS standards.
When a SAS domain starts up, one or more initiator devices perform a “Discovery” process in accordance with the SAS specifications so that each SAS component may generate information indicative of the SAS domain topology. In other words, the SAS Discovery process specified by the SAS specifications permits each SAS device and each SAS expander in the domain to discover information about immediate neighbors coupled to its ports as well as information about other devices and expanders coupled to ports of neighboring components. Thus, each SAS device and SAS expander in a SAS domain may acquire information regarding the overall geometry or topology of the SAS domain.
When a change occurs in the SAS domain, a SAS expander as presently known in the art has only a limited role in adapting devices in the SAS domain to reconfigure for the sensed change. In particular, if a SAS expander senses a change in the SAS domain, as presently known in the art, the SAS expander may issue a BROADCAST(CHANGE) primitive to broadcast to all SAS initiators the fact that some change has occurred. The SAS expander does not inform the SAS initiators what change has been detected. Rather, a SAS initiator that receives the BROADCAST primitive on the SAS communication medium will perform the SAS Discovery process anew. The SAS Discovery process re-discovers all configuration and topological information regarding devices in the SAS domain-whether changed or not changed. Performing a complete SAS Discovery process to detect any change in a SAS domain, even a small change, consumes valuable resources in the SAS initiator and valuable bandwidth in the SAS domain communication paths.
A system relying on SAS expanders can include means to send and receive out of band (OOB) signals over a physical layer. OOB signals can be low-speed signal patterns that do not appear in normal data streams. OOB signals generally consist of defined amounts of idle time followed by defined amounts of burst time. During the burst time, the physical link carries signal transitions. The signals are differentiated by the length of idle time between the burst times. Included in the SATA standard are two predefined OOB signals: COMINIT/COMRESET and COMWAKE. An additional SAS standard-specific OOB signal COMSAS is predefined. The use of these predefined OOB signals, including detection, transmission and reception are defined in section 6.6 of the SAS specification document “Working Draft Serial Attached SCSI-1.1 (SAS-1.1)”, revision 9e, 24 Jul. 2005 and published by T10, a technical subcommittee of the International Committee for Information Technology Standards (INCITS) which document is hereby incorporated herein by reference in its entirety.
SAS expanders can be configured to receive SCSI Enclosure Services (SES) pages (also referred to as SES messages) that set and get the various fault detection parameters, that get the status of phys, and that directly enable or disable individual phys. Control and status information, such as SES pages, may be sent via an OOB communication path between the SAS expanders.